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Abstract: In the present study, an examination is conducted in three school districts of how data are 
used to improve classroom practice. In doing so, we explore the effects that attitudes toward data, 
principal leadership, and computer data systems have on how data are used to affect classroom 
practice. Findings indicate that educators are ambivalent about data: they see how data could 
support classroom practice, but their data use operates in the presence of numerous barriers. Many 
of these barriers are due to principal leadership and computer data systems; these barriers often have 
negative effects on attitudes toward data and dismpt the progression from using data to inform 
classroom practice. It is hypothesized that many of these barriers can be removed through effective 
district policies to improve structures and supports for using data. 
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Efectos generales en los distritos escolares del uso de datos en el aula 
Resumen: En el presente estudio, se examinan como tres distritos escolares usan datos 
para mejorar la practica docente. Se exploraron que efectos tienen las actitudes hacia los 
datos, el liderazgo de los directores/as, y los sistemas de procesamiento de datos para 
modificar practicas en el aula. Los resultados indican que los educadores son ambivalentes 
acerca de los datos: ven como los datos podrian apoyar practicas en el aula, pero el uso de 
datos funciona en presencia de numerosas barreras. Muchas de estas barreras se deben al 
liderazgo de directores/as y a los sistemas de procesamiento de datos informaticos. Estas 
barreras suelen tener efectos negativos en las actitudes hacia los datos e interrumpen la 
incorporation de datos para mejorar practicas en el aula. Se formula la hipotesis de que 
muchas de estas barreras pueden ser removidas a traves de politicas distritales eficaces que 
mejoren las estructuras y brinden apoyos para el uso de datos. 

Palabras clave: uso de information; toma de decisiones basadas en datos; reforma 
educativa. 

Efeitos gerais nos distritos escolares do uso de dados em sala de aula 
Resumo: Neste estudo, examinamos como tres distritos escolares utilizaram dados para 
melhorar a pratica docente. Foram explorados os efeitos de atitudes em rela^ao aos dados, a 
lideranga dos/as diretores/as, e os sistemas de processamento de dados para alterar as praticas 
de sala de aula. Os resultados indicam que os educadores sao ambivalentes respeito a o uso dos 
dados: observam como os dados podem apoiar as praticas de sala de aula, mas o uso de dados se 
faz na presenca de muitas barreiras. Muitas dessas barreiras sao devidas a lideran^a dos/as 
diretores/as e aos sistemas e processamento de dados informaticos. Essas barreiras tendem a ter 
efeitos negativos nas atitudes com o uso dos dados e interrompem os processos de incorporar 
dados para melhorar as praticas de sala de aula. Se formula a hipotese de que que muitas dessas 
barreiras podem ser removidas atraves de politicas distritais eficazes que melhoram as estruturas 
e fornecendo suporte para o uso de dados. 

Palavras-chave: uso de informa^oes; toma decisoes com base em dados; reforma educacional. 

Introduction 

During the last 10 years, the field of education has witnessed a substantial increase in studies 
which examine how educators may use student data to help improve their practice. This research has 
shown a number of factors that facilitate classroom data use, such as collaboration, principal 
leadership, personnel supports, and effective technology (e.g., Anderson, Leithwood, & Strauss, 
2010; Wayman & Stringfield, 2006; Copland, 2003; Lachat & Smith, 2005; Marsh, McCombs, & 
Martorell, 2010). 

Much of the research on educational data use has focused on case studies and empirical 
descriptions. Thus, there are few established causal links between student achievement and the use 
of data (Hamilton, Halverson, Jackson, Mandinach, Supovitz & Wayman, 2009). This 
notwithstanding, there is reason to believe that the effective use of data may improve schooling. For 
instance, the effective use of data is often cited as part of more general school improvement 
initiatives (Ingram, Louis, & Schroeder, 2004; Stringfield & Datnow, 2002), teachers often report 
changes in practice based on data use (Wayman & Stringfield, 2006; Datnow, Park, & Wohlstetter, 
2007), and studies are emerging that statistically correlate student achievement to interim assessment 
administration (Wayman, Shaw & Cho, 2011; Carlson, Borman, & Robinson, 2011; May & 

Robinson, 2007). 
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Still, a set of scalable, effective data use practices remains elusive. Knowledge of effective 
data practice has largely been created by studying contexts chosen for exemplary conditions (e.g., 
Wayman & Stringfleld, 2006; Datnow et al., 2007; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006), 
but studies in more typical contexts reveal persistent problems with using data (Wayman, Cho & 
Johnston, 2007; Wayman, Cho, & Shaw, 2009a; Valli & Buese, 2007; Young, 2006). Further, these 
studies are often narrow in scope; few studies have examined data use throughout an entire district 
(Wayman, et al., 2007) or across multiple districts (Anderson et al., 2010; Datnow et al., 2007). 

Thus, there remains a need for further study of how data use at all levels of a district affect 
classroom practice, and in contexts not chosen for their proclivity in using data. The present study 
responds to that need through an examination of three districts of differing size and context, none 
of which were chosen for their excellence in using data. Accordingly, the goal of this study is to 
examine how practices at every level of the district affect the use of data in the classroom. We focus 
our research questions on four areas that prior research suggests may be particularly important: 

(1) How do educators commonly use data? 

(2) What are educators’ attitudes toward using data? 

(3) How do principals lead faculty in using data? 

(4) How well do computer data systems support educators in using data? 

Research on Educational Data Use 

Educational research has noted the wide variety of factors that influence educational data 
use. In the present study, we examine four areas that arise frequently in research: attitudes toward 
data, principal leadership, and computer data systems. In this section, we provide a brief overview of 
research in each of these four areas, followed by a model that conceptualizes how these factors work 
together to promote effective data use. 

Educators’ use of data. Research on educational data use notes that uses vary by role, but has 
focused primarily on how teachers and principals use data. For instance, research in exemplary 
settings has shown teachers using a variety of student-level data to group students, re-group 
students, and adjust instruction based on data (Wayman & Stringfleld, 2006; Datnow et al., 2007; 
Lachat & Smith, 2005). Research in exemplary settings has shown principals using both student- and 
building-level data to make policy decisions and support faculty (Copland, 2003; Datnow et al., 

2007). Research has provided little detail on data uses of central office staff and instructional 
coaches, with the exception of studies such as Honig and Coburn (2008) and Marsh et al., (2010). 

Educators’ attitudes toivard data. Educators in rich data-using contexts often report that 
educators have positive attitudes toward data when supported by a culture of data use (Wayman & 
Stringfleld, 2006; Copland, 2003; Datnow et al., 2007; Lachat & Smith, 2005; Knapp et al., 2006). 
Educators in these studies typically believed that using data helped them improve practice and 
resulted in improvements in student learning. Educators in other contexts often display more 
tempered attitudes. For instance, teachers have sometimes been shown to be suspicious of data 
initiatives, often separating data use from their own judgment (Ingram et al., 2004; Valli & Buese, 
2007; Young, 2006). Educators in all roles are sometimes hesitant to use data for fear that it will 
require a large time investment and little practical return (Wayman, et al., 2009a). 

Principal leadership. Successful school-based data initiatives are almost always marked by 
principals who are employing practices such as setting clear expectations for data use, involving 
entire faculties, and making time for teachers to use data (Wayman & Stringfleld, 2006; Copland, 
2003; Datnow et al., 2007; Halverson, Prichett, & Watson, 2007; Knapp, Swinnerton, Copland, & 
Monpas-Huber, 2006; Supovitz & Klein, 2003). On the other hand, studies of contexts not known 
for data use describe a wide range of principal involvement (Anderson et al., 2010; Wayman et al., 
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2007; Young, 2006) - some principals in these studies were effectively leading faculty in data use, 
but most were not. Those who were not were often characterized by these studies as either 
disinterested in data use or lacking a set of strategies that could foster effective faculty data use. 

Computer data systems. Nearly every district in the United States has some form of computer 
system for managing student data (Means, Padilla, DeBarger, & Bakia, 2010). When implemented 
effectively, these systems have been shown to facilitate many facets of educator data use, such as 
collaboration and rapid turnaround of data (Wayman & Stringfield, 2006; Lachat & Smith, 2005; 
Long, Rivas, light, & Mandinach, 2008). Other research chronicles the struggles of educators 
unsupported by effective data systems. These studies have shown computer systems often lack 
integration, are inefficient, and are hard to use (Means et al., 2010; Wayman et al., 2007; Wayman et 
al., 2009a). 

Conceptualframework. Many scholars have presented models to describe how the use of data 
may improve education (e.g., Copland, 2003; Mandinach, Honey, Light, & Brunner, 2008; Supovitz, 
2010). While the details of these models differ, they all share the same core logic: data use provides 
information that educators may employ to change practice. 

The research base on educational data use shows mixed results in the application of this 
model. While some studies show this progression to happen successfully (Wayman & Stringfield, 
2006; Datnow et al., 2007; Lachat & Smith, 2005), others show that inappropriate use of data has 
actually hindered educational practice (Wayman, et al., 2009a; Vasquez Heilig & Darling-Hammond, 
2008; Valli & Buese, 2007). 

Prior knowledge, information, and other elements may influence how certain data are 
noticed, prioritized, and used (Alavi & Leidner, 2001; Coburn, Honig, & Stein, 2009; Tuomi, 1999). 
Thus, we posit that the progression from data to knowledge to practice is influenced by a variety of 
elements. The present study focused on three such elements: attitudes toward data, principal 
leadership, and computer data systems. Figure 1 provides a graphic representation of our logic: 




Figure 1. Conceptual Framework of Data Use at the Building Level. 


In this conceptualization, educator use of data provides information that influences classroom 
practice. However, educator uses of data are influenced by an educator’s attitude toward data, 
principal leadership, and the access they gain to data through data systems. It is important to 
distinguish data from information. Data are the raw inputs (e.g., student test scores or teacher 
observations) that educators may access about their students; information is the processed outcome 
of these data. Thus, educators access data from their computer data system and process it into 
information through their uses of data. This information is used to make changes to classroom 
practice. 
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While this is a building-level model, it is deliberately not role-specific, allowing for roles such 
as central office or instructional support to influence how practice occurs in the classroom. For 
example, how an instructional coach’s uses of data influences teachers’ classroom practice will be 
influenced by the coach’s attitude toward data, the leadership of campus principal(s) and the data 
systems that provide access. It is also important to note that the model does not assume that data 
use leads to improved practice - instead, the model allows for the possibility that these or other 
influences might result in data use that leads to more or less effective practice. 

Method 


Introduction 

The present study is drawn from a larger three-year project designed to help three school 
districts improve their use of data by employing a systemic focus called “the Data-Informed District 
(Wayman, 2010; Wayman et al., 2007). It is important to define three terms used in this project. 

First, we use a broad and encompassing definition of data, meaning anything that helps educators 
know more about their students (e.g., formal assessments, tests, quizzes, and student background 
data). Second, data use means the actions in which educators engage as they collect these data, 
organize and analyze them, and draw meaning from them to inform practice. Third, we often use 
the term effective data use to distinguish between data use practices that benefit educators in their 
practice (and which thus benefits student learning) from other data use practices that have been 
shown to actually hinder educational work (Wayman et al., 2007; Earl & Fullan, 2003; Valli & Buese, 
2007; Young, 2006). 

Data were collected in three districts in Texas 1 2 3 4 during the 2009-2010 school year. Districts 
were not selected for their success at using data; in fact, district leaders volunteered for this study to 
improve their districts’ data use. Boyer School District was a district of approximately 8,000 students 
that mostly served a non-Latino White population,- less than five percent of who were economically 
disadvantaged. Gibson School District was a district of approximately 25,000 students of various 
ethnic backgrounds, 1 half of whom were economically disadvantaged. Musial School District was a 
district of approximately 45,000 students of various ethnic backgrounds, a third of whom were 
economically disadvantaged. Districts varied in their student achievement: in a typical year, the 
percent of students meeting standards on the state exam was consistendy greater than 95% in Boyer, 
approximately 75% in Gibson, and approximately 85% in Musial. The overall state rate was typically 
about 82%. 

Procedure 

We employed mixed-methods in conducting this study. Phone and in-person interviews were 
conducted with individuals, site visits were made to schools to conduct educator focus groups, and a 
confidential online survey was made available to all educators in each study district. In the following 
sections, we describe our procedures for collecting the qualitative and quantitative samples. 

Qualitative sample. Qualitative data were collected through individual interviews and focus 
groups. These were conducted using a semi-structured protocol that focused discussion on ways 
data were used and accessed, specific data systems employed, and wishes for future data use. All 
qualitative interviews were recorded and subsequently transcribed for analysis. 


1 Pseudonyms are used for each district. 

2 80% non-Latino White, 10% Latino. 

3 40% Latino, 30% non-Latino White, 20% African American. 

4 50% non-Latino White, 25% Latino, 10% African American. 
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At the central office level, individuals were identified through a review of central office 
positions. This list then was discussed with our primary district contacts to ensure proper coverage. 
Additionally, many interviewees were asked to suggest other individuals to interview. Central office 
educators were interviewed by telephone or in person. 

At the building level, teachers, principals, and other building staff participated through focus 
groups conducted during site visits to 19 campuses. These schools were chosen to be representative 
of each district in terms of size, location, and socioeconomic makeup (see Table 1). 

Table 1 


Study Campuses, By District and Level 



Elementary 

School 

Middle 

School 

High 

School 

Total 

Boyer 

2 

1 

1 

4 

Gibson 

4 

2 

1 

7 

Musial 

5 

2 

1 

8 

Study total 

11 

5 

3 

19 


During school site visits, two focus groups were conducted in each school. One focus group 
consisted of the principal and individuals they chose who were familiar with data use in their school 
(e.g., assistant principals or instructional support staff). On the same day, a teacher focus group was 
conducted in that school. Teacher focus groups included three to five teachers who were selected by 
the principal from a randomly generated list of seven to nine teachers. 1 High schools had many more 
teachers on staff than other schools; to ensure that we fully understood data use in the high schools, 
we conducted two teacher focus groups at those sites. The qualitative sample consisted of 197 total 
participants. Table 2 provides a description of this sample, disaggregated by educational role and 
district. 

Table 2 


Study Participation, By Role and District 



Survey Data 

Interview Data 


N 

N 

Boyer 

Teachers 

284 

22 

Campus Administrators 

13 

11 

Central Office 

3 

6 

Instructional Support 

21 

12 

Boyer Total 

321 

51 


5 This procedure created randomness in selecting teachers, but allowed principals latitude needed to collect a 
group of teachers at the same time (e.g., finding coverage). 
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Table 2 (continued) 


Survey Data 

Interview Data 


N 

N 

Gibson 

Teachers 

1117 

30 

Campus Administrators 

62 

18 

Central Office 

34 

11 

Instructional Support 

82 

6 

Gibson Total 

1295 

65 


Musial 

Teachers 

1215 

37 

Campus Administrators 

52 

17 

Central Office 

72 

16 

Instructional Support 

146 

11 

Musial Total 

1485 

81 


Study total 3,101 197 


Quantitative satnple. Quantitative data were collected by administering the Survey of Educator 
Data Use (Wayman, Cho, & Shaw, 2009b), a 67-item instrument assessing a variety of factors, 
including attitudes toward data use, support for data use, instructional practices, technology, and 
specific ways in which data were used by the respondent. The survey was given online and made 
available to all educators throughout each district. Participants were not allowed to leave any items 
blank, so there were no missing survey data. Survey response rates were 50% in Boyer, 62% in 
Gibson, and 41% in Musial. The quantitative sample consisted of 3,101 individuals across the three 
districts. Table 2 provides a description of this sample, disaggregated by educational role and district. 

Measures 

Comparison categories. Educational role and district experience were each used to compare 
educators on aspects of data use. We defined four roles to be used for comparison: (1) campus 
administrators (principals and assistant principals), (2) central office staff, (3) instructional support 
staff (campus staff such as counselors, school psychologists, and instructional coaches), and (4) 
teachers. On the survey, district experience was collapsed into four categories: (1) 5 years or less, (2) 
6 — 11 years, (3) 11 - 20 years, and (4) 20 years or more. 

Data use questions. Selected survey items were singled out to help describe how district 
educators used data. One block of 14 questions asked how often participants engaged in specific 
data uses, such as identifying individual students who need remedial assistance, setting school 
improvement goals, and evaluating district achievement trends and performance. These items were 
set on a four-point Likert scale with the following options: less than once a month, once or twice a month, 
weekly or almost weekly, and a few times a week. Each response option was numbered 1-4, with 1 
corresponding to less than once a month. 

Another block of questions asked how often participants used specific computer systems in 
their district. These items were set on the same four-point Likert scale as the questions above. 

Four scales measuring attitudes and uses of data were formed from groups of survey items. 
The individual survey items within these groups asked how much the respondent agreed with a 
statement, offering the following options: strongly disagree , somewhat disagree, somewhat agree, and strongly 
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agree. Each response option was numbered 1-4, with 1 corresponding to strongly disagree. To create 
each scale, responses were averaged across the group of items in that scale. Scales thus ranged from 
one to four. These scales have been shown to be valid and reliable in other samples (Wayman et al., 
2007; Wayman et al., 2009a). 

The Attitudes Toivard Data scale was a four-item scale that asked participants whether they 
liked data, found it useful, and whether it helped them. The alpha reliability of this scale ranged from 
0.838 to 0.893 6 . The Data's Effectiveness for Pedagogy scale consisted of five items that asked about the 
contributions that data can make for improving educational practice (e.g., helping to plan 
instruction, reveal new insights, or identify learning goals). The alpha reliability of this scale ranged 
from 0.930 to 0.949. The PrincipalEeadership scale consisted of five items that asked how much 
participants agreed that their principal or assistant principal(s) supported data use (e.g., encouraged it 
as a tool to support teaching, made training available, were examples of effective data users). The 
alpha reliability of this scale ranged from 0.909 to 0.910. The Computer Data Systems scale consisted of 
four items asking about computer systems, such as whether the participant’s systems were easy to 
use or provided ample data. The alpha reliability of this scale ranged from 0.943 to 0.951 

Analyses 

Qualitative analyses followed methodology suggested by Miles and Huberman (1984). 
Drawing upon prior research on educational data use, an a priori list of potential analytic themes was 
generated. As qualitative data collection progressed, these themes were updated and refined during 
research team meetings. This collaborative and inductive process resulted in a conceptually coherent 
set of themes that was used for coding interviews and focus groups. The research team used this set 
of themes to code participant responses. Themes were examined by role and school level to identify 
emergent patterns and explanations regarding educator data use. 

Quantitative analyses were conducted as follows: for the block of 14 questions that asked 
about uses of data, mean responses were ranked for each role. This produced a ranked list of the 
frequency of data use for each of the 14 questions. For the four survey scales, ANOVAs were used 
to compare means responses by role. Significance was set at the 0.05 level and effect sizes (partial 
eta-squared) were computed. When role was statistically significant, Tukey post-hoc tests were 
performed to identify significant mean differences. Due to space restrictions, simultaneous 
confidence intervals for mean differences were computed but are not presented in this paper. For 
the questions that asked about frequency of computer use, we classified each system as an 
assessment system, student information system, data warehouse, or “other” system and 
dichotomized responses into weekly use or less. Percentages of weekly use were presented by role 
for each type of system. 


Results 

Besides differences in demographics, size, and economic makeup, our districts also 
presented diverse contexts in the ways data were approached and used. In the Boyer school districts, 
“data use” was viewed by most educators as the examination of state test data. Since almost every 
Boyer student passed the state test, many educators believed the state tests were irrelevant to their 
work - and thus, so was data use. Consequently, efforts to use data in Boyer were confined to small 
groups of interested educators. The district was attempting to implement some procedures around 
formative assessments, but was having trouble building momentum. 


6 For each scale, alpha reliability was calculated specific to each district’s survey results. 
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In Gibson, state test performance was strongly emphasized, but as part of a larger, 
curriculum-based initiative. Curriculum was divided into segments and a locally-built benchmark 
examination was given to students at the end of each segment. These examinations were tied to 
curriculum but were also intended to prepare students for performance on state tests. 

In Musial, there was a very strong focus on state test performance. District leadership used a 
variety of ways to communicate the importance of state test performance. Musial also implemented 
a set of locally-built assessments tied to state exams. Teachers were expected to examine these 
periodic assessments to improve instruction, with an eye toward performance on state tests. Musial 
had recently hired a central office administrator whose role was to support data use throughout the 
district with a strong focus on working with building-level educators on using data to improve 
instruction. 

In the following, we provide analysis of how these districts used data in their varying 
contexts. Sections are provided corresponding to our four research questions: (1) uses of data, (2) 
attitudes toward data, (3) principal leadership for data use, and (4) computer data systems. 

Uses of Data 

The ways data were used varied by role in all three districts. We use the following sections to 
provide descriptions of how data were used in four roles: central office educators, instructional 
support specialists, campus administrators, and teachers. 

Central office educators. Central office educators, in line with their responsibilities for large 
numbers of students and teachers, tended to use data for monitoring district and campus progress, 
providing feedback to campus personnel, and in support of broad-scale campus efforts. Many of 
these uses were centered on attempts to help building-level educators improve practice. For 
example, Gibson and Musial central office educators provided feedback to campuses about 
particular goals; campus personnel were expected to use this feedback to make adjustments in areas 
identified for improvement. As another example, central office educators in Boyer performed item 
analysis on behalf of teachers and helped inform departments and teachers about academic areas in 
which various grade levels needed attention. 

Instructional support specialists. Despite a variety of titles and intended responsibilities, the ways 
that instructional support specialists used data were consistent across the three districts. Persons in 
instructional support roles indicated that they used data in three main ways. First, they used data to 
identify and help teachers address the needs of individual students. Tables 3 through 5 show that 
instructional support personnel in both Musial and Boyer ranked the use of data to identify student 
learning first (M=2.91 and M=3.14, respectively); in Gibson, it was a close second (M= 2.70). 
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Table 3 


Respondent Rankings of Data Uses in Boyer, By Role 



Teachers 


Campus Administrators 


Instructional Support 

1 . 

Tailor instruction to individual 

1 . 

Develop recommendations for 

1 . 

Identify learning needs of 


student needs (2.46) 


tutoring or other educational 


students who are struggling 




services for students (2.62) 


(3.14) 

2. 

Identify learning needs of 

2. 

Identify learning needs of 

2. 

Set learning goals for individual 


students who are struggling 


students who are struggling 


students (3.05) 


(2.46) 


(2.46) 



3. 

Identify instructional content to 

3. 

Form small groups of students 

3. 

Discuss student progress or 


use in class (2.33) 


for targeted instruction (2.38) 


instructional strategies with 
other educators (2.95) 

4. 

Set learning goals for individual 

4. 

Meet with a specialist about 

4. 

Develop recommendations for 


students (2.31) 


data - e.g., instructional coach 


tutoring or other educational 




(2.31) 


services for students (2.81) 

5. 

Discuss student progress or 

5. 

Discuss student progress or 

5. 

Assign or reassign students to 


instructional strategies with 


instructional strategies with 


classes or groups (2.71) 


other educators (2.28) 


other educators (2.23) 



6. 

Form small groups of students 

6. 

Tailor instruction to individual 

6. 

Tailor instruction to individual 


for targeted instruction (2.23) 


student needs (2.23) 


student needs (2.71) 

7. 

Develop recommendations for 

7. 

Set learning goals for individual 7. 

Identify learning needs of 


tutoring or other educational 


students (2.23) 


students who are not struggling 


services for students (2.12) 




(2.62) 

8. 

Assign or reassign students to 

8. 

Assign or reassign students to 

8. 

Discuss data with a parent 


classes or groups (2.08) 


classes or groups (2.15) 


(2.52) 

9. 

Identify learning needs of 

9. 

Discuss data with a parent 

9. 

Identify instructional content to 


students who are not struggling 
(2.06) 


(2.15) 


use in class (2.48) 

10. 

Discuss data with a parent 

10. 

Discuss data with a student 

10. 

Form small groups of students 


(1.81) 


(1.92) 


for targeted instruction (2.48) 

11. 

Discuss data with a student 

11. 

Choose which parents to 

11. 

Meet with a specialist about 


(1.79) 


contact (1.85) 


data - e.g., instructional coach 
(2.33) 

12. 

Choose which parents to 

12. 

Identify learning needs of 

12. 

Choose which parents to 


contact (1.76) 


students who are not struggling 

contact (2.24) 




(1.85) 



13. 

Meet with a specialist about 



13. 

Interact with your principal 


data - e.g., instructional coach 
(1.75) 




about data use (2.10) 

14. 

Interact with your principal 



14. 

Discuss data with a student 


about data use (1.45) 




(2.10) 


Note. Mean response is shown in parentheses and only uses specific to the role are included. 
Note. Central office is not included because few uses applied to that specific role. 

Note. Teacher: n=284. Campus administrator: n=13. Instructional support: n=21. 
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Table 4 


Respondent Rankings of Data Uses in Gibson, By Role 


Teachers 


Campus Administrators 

Instructional Support 

1. Identify learning needs of 

1. 

Identify learning needs of 

1. Discuss student progress or 

students who are struggling 


students who are struggling 

instructional strategies with 

(2.72) 


(3.16) 

other educators (2.74) 

2. Tailor instruction to 

2. 

Discuss student progress or 

2. Identify learning needs of 

individual student needs 


instructional strategies with 

students who are struggling 

(2.68) 


other educators (3.13) 

(2.70) 

3. Identify instructional content 3. 

Develop recommendations 

3. Tailor instruction to 

to use in class (2.60) 


for tutoring or other 

individual student needs 



educational services for 
students (3.03) 

(2.65) 

4. Form small groups of 

4. 

Tailor instruction to 

4. Set learning goals for 

students for targeted 


individual student needs 

individual students (2.55) 

instruction (2.51) 


(2.90) 


5. Set learning goals for 

5. 

Meet with a specialist about 

5. Develop recommendations 

individual students (2.50) 


data - e.g., instructional 

for tutoring or other 



coach (2.84) 

educational services for 
students (2.55) 

6. Develop recommendations 

6. 

Set learning goals for 

6. Identify instructional content 

for tutoring or other 
educational services for 


individual students (2.73) 

to use in class (2.50) 

students (2.47) 

7. Discuss student progress or 

7. 

Form small groups of 

7. Assign or reassign students 

instructional strategies with 


students for targeted 

to classes or groups (2.39) 

other educators (2.45) 


instruction (2.69) 


8. Assign or reassign students 

8. 

Discuss data with a parent 

8. Interact with your principal 

to classes or groups (2.35) 


(2.65) 

about data use (2.34) 

9. Identify learning needs of 

9. 

Assign or reassign students 

9. Form small groups of 

students who are not 


to classes or groups (2.61) 

students for targeted 

struggling (2.28) 



instruction (2.34) 

10. Discuss data with a student 

10. Discuss data with a student 

10. Meet with a specialist about 

(2.12) 


(2.58) 

data - e.g., instructional 

11. Choose which parents to 

11. 

, Choose which parents to 

11. Identify learning needs of 

contact (1.91) 


contact (2.50) 

students who are not 
struggling (2.12) 

12. Meet with a specialist about 

12. Identify learning needs of 

12. Discuss data with a student 

data - e.g., instructional 


students who are not 

(2.12) 

coach (1.83) 

13. Discuss data with a parent 


struggling (2.29) 

13. Discuss data with a parent 

(1.83) 



(2.04) 

14. Interact with your principal 



14. Choose which parents to 

about data use (1.66) 



contact (1.94) 


Note. Mean response is shown in parentheses and only uses specific to the role are included. 
Note. Central office is not included because few uses applied to that specific role. 

Note. Teacher: n=1117. Campus administrator: n=62. Instructional support: n=82. 
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Table 5 


Respondent Rankings of Data Uses in Musial By Role 



Teachers 


Campus Administrators 


Instructional Support 

1. 

Identify learning needs of 

1. 

Identify learning needs of 

1. 

Identify learning needs of 


students who are struggling 


students who are struggling 


students who are struggling 


(2.80) 


(3.37) 


(2.91) 

2. 

Tailor instruction to individual 

2. 

Develop recommendations for 2. 

Discuss student progress or 


student needs (2.76) 


tutoring or other educational 


instructional strategies with 




services for students (3.31) 


other educators (2.81) 

3. 

Identify instructional content to 

3. 

Discuss student progress or 

3. 

Develop recommendations for 


use in class (2.62) 


instructional strategies with 


tutoring or other educational 




other educators (3.29) 


services for students (2.73) 

4. 

Discuss student progress or 

4. 

Set learning goals for individual 4. 

Tailor instruction to individual 


instructional strategies with 
other educators (2.57) 


students (3.17) 


student needs (2.66) 

5. 

Set learning goals for individual 

5. 

Discuss data with a parent 

5. 

Set learning goals for individual 


students (2.54) 


(3.10) 


students (2.66) 

6. 

Form small groups of students 

6. 

Tailor instruction to individual 

6. 

Interact with your principal 


for targeted instruction (2.54) 


student needs (3.02) 


about data use (2.32) 

7. 

Develop recommendations for 

7. 

Discuss data with a student 

7. 

Discuss data with a student 


tutoring or other educational 
services for students (2.53) 


(2.98) 


(2.29) 

8. 

Assign or reassign students to 

8. 

Form small groups of students 

8. 

Assign or reassign students to 


classes or groups (2.38) 


for targeted instruction (2.98) 


classes or groups (2.23) 

9. 

Identify learning needs of 

9. 

Choose which parents to 

9. 

Form small groups of students 


students who are not struggling 
(2.38) 


contact (2.98) 


for targeted instruction (2.22) 

10. 

Discuss data with a student 

10. 

Assign or reassign students to 

10. 

Discuss data with a parent 


(2.25) 


classes or groups (2.96) 


(2.21) 

11. 

Choose which parents to 

11. 

Meet with a specialist about 

11. 

Identify instructional content to 


contact (2.25) 


data - e.g., instructional coach 

(2.94) 


use in class (2.10) 

12. 

Discuss data with a parent 

12. 

Identify learning needs of 

12. 

Meet with a specialist about 


(1.92) 


students who are not struggling 

data - e.g., instructional coach 




(2.81) 


(2.10) 

13. 

Interact with your principal 



13. 

Identify learning needs of 


about data use (1.78) 




students who are not struggling 
(2.02) 

14. 

Meet with a specialist about 



14. 

Choose which parents to 


data - e.g., instructional coach 
(1.78) 




contact (1.92) 


Note. Mean response is shown in parentheses and only uses specific to the role are included. 
Note. Central office is not included because few uses applied to that specific role. 

Note. Teacher: n=1215. Campus administrator: n=52. Instructional support: n=146. 





District-Wide Effects on Data Use in the Classroom 


13 


Second, instructional support personnel used data to collaborate with teachers. Interview 
data revealed that instructional support personnel used a variety of data (e.g., primary reading data, 
math inventories) to aid teachers in forming small groups of students or to help teachers prepare 
lessons that target specific skills. Further, Tables 3 through 5 show that discussing student progress 
or instructional strategies ranked high in survey data: it was the highest-ranking survey variable in 
Gibson (M=2.74), second for Musial (M=2.81), and third for Boyer (M=2.95). 

Third, interview data suggested that instructional support personnel used data to help 
teachers reflect on practice. This included both monitoring and diagnosing of aggregated groups, as 
well as intervention and support with individual teachers and students. 

Campus administrators. In each district, our data showed that campus administrators often 
focused their data use on struggling students. This issue came up first and frequently in nearly every 
administrator focus group. On the survey, campus administrators ranked the use of data for 
identifying the needs of struggling students and for developing recommendations for intervention as 
the top two most frequent uses of data (see Tables 3 through 5). Administrators reported far less 
frequent use of data to identify the learning needs of students who were performing adequately or 
beyond, ranking this use last among all survey options in each district (see Tables 3 through 5). 

In addition, administrators also reported some frequent non-student based issues to which 
they applied data. One such use involved teacher feedback and evaluation efforts: administrators in 
Gibson and Musial described classroom “walkthroughs” and how they collected and reported data 
from this process. Administrators in each study district also described using various data to gauge 
the fidelity of curriculum implementation (i.e., whether teachers were on schedule and assessing the 
rigor of their teaching strategies). 

Teachers. Across the districts, teachers reported a variety of uses of data, including using data 
to help struggling students, group and regroup for instruction, reteach particular concepts and skills, 
and adjust instruction. Similar to administrators, teachers in interviews often focused on using data 
to support struggling students. This use also ranked first or second for surveyed teachers in each 
district (see Tables 3 through 5). Teacher comments less frequently focused on the needs of students 
who were performing adequately or who were excelling in the classroom. Such use of data ranked 
ninth among teachers in all three districts (see Tables 3 through 5). 

Teachers also discussed various ways that they used data to change how or what they taught; 
Tables 3 through 5 show these items rank highly. In interviews, this often related to grouping, such 
as forming instructional groups of students or deciding what to teach to the entire group. Teacher 
comments in each district indicated that their attention to groups occurred infrequently during the 
year, such as at the start of quarters. Although surveyed teachers reported frequently using data to 
adjust instruction for individual students (see Tables 3 through 5), we heard little mention of this in 
focus groups. 

We also found that teachers only rarely talked about using data as a centerpiece of meetings 
with instructional support personnel or with campus administrators. Such use of data ranked near 
the bottom for surveyed teachers in each district (see Tables 3 through 5). In focus groups, teachers 
either did not mention this type of data use or described it as happening infrequently. 

Attitudes Toward Data 

Survey and interview data revealed that participants were generally positive about data and its 
potential, even in the face of consistently present barriers. These barriers made many educators 
ambivalent about data - they saw value in data, but were hesitant because of hardships and other 
concerns. In the following two sections, we present results describing positive attitudes held by 
participants, followed by results that outline the barriers that temper these attitudes. 
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Positive attitudes. In each district, we found educators to be generally positive about data and 
what it could do for their practice. Participants in all roles averaged at least 3.00 on the Attitudes 
toivard Data and Data’s Effectiveness for Pedagogy scales, with some roles approaching the maximum of 
4.00. In interviews, participants often spoke positively of data and their potential. 

While positive, teachers often displayed more skepticism about data than did those in other 
roles. Table 6 shows that teachers’ views were significandy different from those in other roles on the 
two scales. Table 7 shows that in each district, teachers ranked significantly lower on the two scales 
(excepting Musial central office administrators, who were similar to teachers on both scales). 

Campus administrators and instructional support personnel consistently ranked high, if not 
significantly so. In fact, campus administrators in Gibson and Musial averaged near the maximum 
on the Effectiveness scale. Educators in all roles showed slightly more optimism about the 
effectiveness of data than in their personal attitudes toward using it. 

Table 6 

One-way ANOVAsfor Survey Scales by Pole, for Each District 

df_F_p_Eta-squared 


Boyer 


Attitudes Toward Data 
Data’s Effectiveness for 

2,315 

9.91 

0.00 

0.06 

Pedagogy 

2, 315 

8.48 

0.00 

0.05 

Principal Leadership 

2, 315 

2.59 

0.08 

0.02 

Computer Data Systems 

2, 315 

0.14 

0.88 

0.00 


Gibson 


Attitudes Toward Data 

3,1291 

35.74 

0.00 

0.08 

Data’s Effectiveness for 
Pedagogy 

3,1291 

25.90 

0.00 

0.06 

Principal Leadership 

3,1291 

8.13 

0.00 

0.02 

Computer Data Systems 

3,1291 

8.80 

0.00 

0.02 


Musial 

Attitudes Toward Data 

3,1484 

36.17 

0.00 

0.07 

Data’s Effectiveness for 
Pedagogy 

3,1484 

20.54 

0.00 

0.04 

Principal Leadership 

3,1484 

31.70 

0.00 

0.06 

Computer Data Systems 

3,1484 

0.69 

0.56 

0.00 


Note. Independent variable is role. 

Note. Central Office administrators are not included for Boyer because of insufficient response. 
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Table 7 

Means of Survey Scales, Disaggregated by District and Role 



Attitudes 
Toward Data 

Data’s 

Effectiveness for 
Pedagogy 

Principal 

Leadership 

Computer 

Data 

Systems 

Boyer 

Teachers 

3.05 ca,is 

3.34 is 

2.97 

3.11 

Campus Administrators 

3.44' 

3.68 

2.89 

3.02 

Central Office 

N/A 

N/A 

N/A 

N/A 

Instructional Support 

3.56' 

3.82' 

3.30 

3.13 


Gibson 

Teachers 

3.13 co ’ ca ’ ,s 

^ 42 co ’ ca ’ is 

3.21 co,is 

3.07 ca,co 

Campus Administrators 

3.71' 

3.92' 

3.35 co 

2.83 is,t 

Central Office 

3.71' 

3.84' 

2.86'’ ca,is 

2.57 is,t 

Instructional Support 

3.53' 

3.69' 

3.41 co,t 

3.12 ca,co 


Musial 

Teachers 

3.12 ca,is 

3.45 caiis 

3.26 ca,co ’ is 

2.94 

Campus Administrators 

3.84'- is 

3.90' ,CO 

3.53' ,co 

3.02 

Central Office 

3.03 ca,is 

3.47 ca>is 

2.62'’ ca,is 

3.00 

Instructional Support 

3.50** 

3.73' ,co 

3.45' ,co 

3.02 


Note. Means are not presented for Boyer Central Office because of insufficient response. 

Note. Significant pairwise role differences are noted by superscripts within the table, t = teachers, ca = 
campus administrators, co = central office, and is = instructional support 


Interview data captures the details of these positive attitudes, which differed by district. 
Educators in Gibson and Musial had greater exposure to data and showed attitudes more similar to 
each other than to Boyer educators. Throughout Gibson and Musial, participants who were positive 
about data saw it as a way to support professional judgment. For them, data contributed to 
instruction (e.g., providing feedback about individual students, lessons, programs, or learning issues) 
and was an important part of reflecting collaboratively about issues of mutual concern. A Musial 
teacher described this attitude thusly: “Say you administer a common assessment, you go to team 
meeting and talk about it: the strengths you are seeing, the weaknesses. ‘[We discuss] How can we 
change our instruction to make this concept more clear?”’ 

In comparison, Boyer educators described their positive attitudes more vaguely than 
educators in Gibson and Musial. Boyer educators spoke generally, as evidenced by this teacher: 
“Working with another grade level teacher [on student data] before the school year is valuable. I get 
to learn what helps certain students.” Further, Boyer administrators typically focused their positive 
attitudes not on themselves, but on benefits for other educators. For example, central office 
administrators valued data as a tool for teachers and campus administrators, but they rarely reported 
using data in their own work. Similarly, campus administrators were positive about data but focused 
on the work of teachers, not themselves. 

Barriers temperingpositive attitudes. The positive attitudes above were often tempered by day-to- 
day difficulties in using data, such as problems with computer systems, lack of time to reflect on 
data, and the labor-intensiveness of using data. In line with survey results, teachers in all three 
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districts described these barriers with more negativity than did educators in other roles. In Gibson 
and Musial, where data were in more frequent use, educators described barriers in greater detail and 
breadth than did Boyer educators. 

Teachers in Gibson and Musial expressed concerns about the role that data was playing in 
their district; these concerns were not seen in Boyer. For example, some Musial teachers felt that 
data were being used to inappropriately compare or encourage unnecessary competition among 
campuses. One said, “We are pressured to meet standards, pressured by the data to meet standards, 
absolutely. They make it clear that that’s very published and very public.” 

While teachers were most vocal about data barriers, non-teachers (campus administrators, 
central office educators, and instructional support educators) also shared these concerns. For 
example, non-teachers in Gibson were especially concerned about their lack of integrated computer 
data systems. They were concerned about challenges in accessing the right data and in sometimes 
having to rely on others to get data for them. As another example, non-teachers in Musial also 
expressed concerns about the difficulties of data access. Additionally, they worried about the kinds 
of conclusions that might be drawn from data, such as data only serving to confirm expectations, 
rather than expanding knowledge. A few also felt that data they had personally collected were more 
informative to decision making. Non-teachers in Boyer were concerned about two issues. One was 
lack of time - particularly, how labor intensive data use could be due to the lack of integration in 
their data systems. The other was a general concern that some educators might undervalue data’s 
role in improving practice since the district already had high levels of student achievement. 

Principal Leadership for Data Use 

Principals across the districts seemed to hold the benefits of data use in high regard (see 
Attitudes Toward Data above). Further, Table 7 suggests that participants across all three districts were 
generally positive about principal leadership for data use, as indicated by averages on the Principal 
Leadership scale. These views varied by role in Gibson and Musial (see Table 6), where more 
expectations were placed on principals to use data. In both districts, principals and their instructional 
support staff scored significantly higher on the Principal Leadership scale than did teachers and central 
office administrators. Central office administrators in both districts scored significantly lower on this 
scale than all other roles. 

Nevertheless, qualitative data show that faculty stmggles with data use were often connected 
to the leadership of their principals. These data show considerable variation in principals’ leadership 
behaviors for data use. Further, we observed principals in Gibson and Musial more involved in 
leading for data use than were principals in Boyer. In the following two sections, we present detail 
on the positive and negative leadership approaches observed. 

Positive leadership strategies. A few principals in our study had established structures that 
promoted regular, consistent data use in their schools. With the exception of one principal in Boyer, 
these principals worked in Gibson or Musial, and were more common at the elementary than 
secondary level. 

A few principals were particularly active in developing robust collaborative routines. Not 
only did they support teacher-to-teacher collaboration, they also worked direcdy with teachers on 
data-related activities and used a collaborative, collegial style in setting expectations and plans for 
using data. Their teachers reported planning with their administrators at team meetings and 
described their administrators as committed to communicating with teachers about data. An 
exchange between Musial focus group teachers described this perspective: 

Teacher 1: He’ll come and sit down with a team. 

Teacher 2: He’ll pare it back for you. 
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Teacher 1: Yeah, he’ll sit down and say, “Well, let’s talk about this and how it 
works.” He wants to know how we’re doing tilings, what’s the growth, he’s very 
serious about that. 

Similarly, a handful of campuses stood as examples for how administrators and teachers 
could be in strong agreement about how and why data can contribute to improving schooling. 
Notably, we heard talk at these campuses that suggested data use was a substantive part of everyday 
work, fostered by leaders and sustained by the interactions and activities of faculty. 

Examples of other structures included explicit time set aside for faculty to examine data use, 
varied forms of collaboration (e.g., grade-level teams, vertical teams, interdisciplinary teams), and 
instructional coaches trained to support teachers in using data. Further, principals that appeared to 
be more skilled at using data were better able to help faculty learn to use data systems, lead 
discussions on the meaning of data, and promote effective questioning. 

Negative leadership strategies. Unfortunately, many more principals were employing few of these 
strategies, or were using strategies that had negative effects. In these schools, teachers directly 
expressed frustration with their administrators’ attempts to lead for data use - their negativity often 
was expressed first toward data explicitly, but frequently transitioned toward comments critical of 
their principals’ leadership behaviors. 

Many teachers were critical of the data skills of their principals and some said their principals 
did not engage in any significant data analysis. One Gibson teacher described how this affected 
faculty data use: 

There are leaders on this campus who don’t know how to analyze the data. And so 
you have pockets of teachers who really good at using and analyzing data, and then 
you have pockets of teachers who just follow along with what those folks are doing, 
but they have no idea how to access any of their data. 

Some principals and faculty described a directive, “top down” approach to leading the use of 
data. Many of these campuses were quite involved in using data, but there was little evidence of 
collegiality. In fact, teachers at these campuses more frequently spoke of data use as a monitoring 
exercise, as this Gibson teacher did: “They want ‘proof that we’re analyzing data - they don’t just let 
me go analyze my data. I don’t need to write anything down, I need to absorb it. In my opinion, they 
want stuff written down so it’s proof that I’ve looked at the data.” 

We also heard of leadership behaviors that seemed punitive in nature. For instance, teachers 
at one Musial campus said that their principal posted lists of teachers whose failure rates were 
greater than ten percent. Another example came from a Gibson campus, where teachers reported 
being called in to discuss data with their administrators when assessment scores were not up to par; 
these teachers reported feeling that the tone of these discussions was punitive. 

Our data revealed few instances in any district of principals structuring time for faculty to 
examine and reflect upon data. There was no evidence that principals were limited by district policies 
in their ability to provide time for data. In fact, there were instances where district policy enabled 
faculty collaborative time (e.g., PLCs, common planning time), but principals rarely specified that 
their faculties spend that time using data. As a consequence, it was common for teachers at all three 
levels to describe their use of data occurring before school, after school, at lunch, or on their 
conference period. Further, there were few examples where principals provided structure for how 
the time set for using data could be spent (e.g., guiding questions, analysis protocols). 

Teachers often reported mixed feelings about their interactions with administrators. In 
Gibson and Boyer, some teachers worried that administrators did not respect other demands on 
teachers’ time. This was especially so for teachers who were asked to perform burdensome and 
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redundant activities, such as hand copying data in preparation for meetings or creating paper records 
for administrators as “evidence” of collaboration. In Musial some teachers were concerned that 
collaborative data events were more about creating “lockstep” and predetermined practices, rather 
than about innovating solutions for students. 

Finally, we were struck by the disconnect we observed between teachers and principals at 
many schools in each district. At some schools, it felt as if administrators and teachers were talking 
about two completely different schools. For example, we frequently encountered schools where 
administrators cited clearly articulated expectations for using data, but teachers were unable to 
articulate these same expectations. As another example, administrators in one Gibson school 
described themselves and their faculty as avid users of data and provided many examples and 
artifacts showing how their school used data. Flowever, teachers at this school were negative toward 
data, reported using it infrequently, and were suspicious of their principal’s motives around using 
data. 

Computer Data Systems 

We defined a “data system” as any computer-based tool that helped educators examine 
student data. Under this definition, the data revealed a diversity of systems employed in each district: 
39 distinct systems were found in Boyer, 58 were found in Gibson, and 68 were found in Musial. 

While the variety of systems was large, many provided data that was fed into to three general 
types of systems: (1) student information systems (SISs), which typically handle day-to-day student 
information, such as scheduling, course grades, and demographic information; (2) assessment 
systems, which rapidly organize test results; and (3) data warehouses, which integrate data from a 
variety of systems and provide a comprehensive, longitudinal view of student performance. 

In the following, we first provide three sections presenting an overview of data systems in 
each district. Next, we offer a section that describes the uses of these systems, followed by a section 
outlining issues in integrating these systems. Finally, we provide a section that outlines educator 
attitudes toward these systems. 

Systems in Boyer. Of the study districts, Boyer was the least advanced in its data technologies. 
Their SIS was not technically an SIS, but an electronic gradebook built to handle grades and other 
student data. Boyer employed two separate assessment systems: one geared toward collecting, 
analyzing, and reporting district- or state-level test results (e.g., district benchmarks and state 
achievement tests), and the other geared toward providing regular, formative assessment data to 
teachers. Boyer had no data warehouse to integrate various systems. 

Systems in Gibson. Gibson employed an SIS for day-to-day information, as well as an 
assessment system for providing assessment data. There existed a data warehouse in Gibson, but it 
was not user-friendly in the ways that it integrated or presented data; consequently, the data 
warehouse typically was used only to organize and access district- and state-level tests. 

Systems in Musial Musial also employed an SIS for day-to-day information and an assessment 
system for providing assessment data. Although Musial technically had a data warehouse, it did not 
exist for many educators in a practical sense: due to site license issues, most educators (and all 
teachers) did not have direct access to the data warehouse. The data warehouse provided reports 
that pulled data from many disparate systems, but access problems had created bottlenecks in 
accessing these reports. Further, when reports arrived, they typically were in the form of a large 
Access or Excel file. 

System uses. Table 8 shows the number and percent of educators that reported weekly use of 
their district’s various systems. In each district, the SIS was the most prevalently used system. 
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Otherwise, these numbers reflect the diversity of technological contexts in the three districts - no 
clear patterns were apparent across district or role. 

Table 8 

Number and Percent of Participants Using Data Systems Weekly or More, Disaggregated by District and Pole 




Assessment 

Data 

Other 


SIS 

System 

Warehouse 

System 

Boyer 

Teachers 

258 (91%) 

64 (23%) 

N/A 

49 (17%) 

Campus Administrators 

11 (85%) 

3 (23%) 

N/A 

2 (15%) 

Central Office 

N/A 

N/A 

N/A 

N/A 

Instructional Support 

14 (67%) 

3 (14%) 

N/A 

5 (24%) 


Gibson 

Teachers 

479 (43%) 

161 (14%) 

138 (12%) 

177 (16%) 

Campus Administrators 

46 (74%) 

10 (16%) 

30 (48%) 

33 (53%) 

Central Office 

13 (38%) 

4 (12%) 

11 (32%) 

14 (41%) 

Instructional Support 

27 (33%) 

17 (21%) 

30 (37%) 

18 (22%) 


Musial 


Teachers 

1097 (90%) 

120 (10%) 

54 (4%) 

211 (17%) 

Campus Administrators 

34 (65%) 

25 (48%) 

20 (39%) 

25 (48%) 

Central Office 

22 (31%) 

7 (10%) 

13 (18%) 

31 (43%) 

Instructional Support 

47 (32%) 

35 (24%) 

13 (9%) 

57 (39%) 


Note: Percent of participants reporting weekly or more use given in parentheses. 

Note. Boyer does not have a data warehouse 

Note. Figures are not presented for Boyer Central Office because of insufficient response. 

Although the distinctions between SISs, assessment systems, and data warehouses may seem 
clear, we found that districts sometimes attempted to “stretch” the intended use of their systems to 
meet locally desired ends. Thus, systems might be used in ways for which they were not necessarily 
designed. For example, the Boyer electronic gradebook was being used by central office members as 
an SIS and doubled for some teachers as both an SIS and gradebook. The Gibson data warehouse 
provided another example: although it could provide data integration, it typically was applied as an 
assessment system. 

We heard some instances of data systems facilitating work, but much more commonly heard 
how technical limitations suppressed data use. For example, many Musial educators described the 
“information overload” they felt with the large Excel files or stacks of printouts they received. 
Gibson educators often experienced a slow turnaround time for feedback regarding district-based 
assessments. Boyer educators rarely described using their systems in ways that were not compliance 
oriented (e.g., state achievement reports or student documentation). 

System integration. In every district, many educators described the desire for easy to access, 
integrated information about students. They wanted the various systems in their district to connect 
in a way that offered one-stop access to all data, as described by a Musial principal: “This district 
wants to be data driven. They value it. They have tons of data, but not one good solid system.” 
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District administrators in each district recognized this problem and each district was 
addressing it in a different way. Boyer administrators were considering a data warehouse and saw 
participating in our study as one way in which they might prepare themselves for the “next level” in 
data system implementation. Gibson was developing plans to obtain a more dynamic data 
warehouse that could provide more refined, comprehensive, and timely views on students. Musial 
was working to expand licenses to allow teachers access to the data warehouse. This work included 
developing a user-friendly system interface, as well as tasking a central office administrator to help 
ensure that educators’ interests were represented in the new interface. 

Educators in each district were going to great lengths to compensate for lack of integration. 
Several educators in each district described using self-built and maintained systems (e.g.. Excel, 
Access, or paper and pencil) in order to aggregate and analyze their students’ data. Although more 
work-intensive and less powerful than the output from an integrated system, the resultant analyses 
were often described as useful. A Gibson assistant principal described one such self-built system 
used in his school: 

Every teacher has a folder and a corresponding spreadsheet that has every single one 
of their students on it. Each student has their ethnicity next to their name, along with 
the class period that they’re in. And what this allows the teacher to do is enter the 
data after an assessment is taken, and the teacher will then be able to sort through 
this to see how students in different classes did by ethnicity and (standard). And then 
they’ll be able to target their intervention by specific (standard) in a special 
intervention period, where the students go to a particular intervention based upon 
need. So it’s a quick way for teachers to have an at-a-glance look at how the kids are 
doing on what they’re teaching in the classroom. 

We uncovered these self-designed systems at every level of every district, sometimes shared across 
teams or buildings via email or a school common drive. In some cases, “power users” were assigned 
the task of querying, assembling, and distributing these makeshift approaches to integrating data. 

Attitudes toivard data systems. Table 6 shows that scores on the Computer Data Systems scale only 
differed by role in Gibson. Table 7 shows that teachers and instructional support staff in Gibson 
perceived their data systems more positively than did campus administrators and central office 
personnel. 

Interview data in all three districts revealed stronger negative attitudes than did survey data. 
Typically, educators in all three districts were highly frustrated with the amount of time it took to 
access and analyze data, and most of their frustration resulted from a lack of system integration. 
Lacking an integrated data system, educators in each district reported that the time burdens of 
generating and compiling data left little time to actually use the data. For example, educators in each 
district reported spending too much time pulling together data from disparate systems and some 
reported hand-copying data to ready it for use. This was especially so in Gibson and Musial, where 
some described using their time to fill out forms that could otherwise have been pre-populated by 
computer data systems. One Gibson teacher described one such example, saying, “We took the [data 
system] printout and copied the same information in a watered-down format onto another sheet of 
paper ... yes, it divided tilings I was strong in and weak in, but I could have done that on the 
printout.” Educators in every district saw such labor-intensive routines as untenable in light of 
today’s advancements in technology. 
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Discussion 

The goal of the present study was to examine how practices at every level of the district 
affect the use of data in the classroom. We addressed this goal by exploring uses of data, attitudes 
toward data, principal leadership, and computer data systems. Educators were ambivalent in their 
attitudes toward data - they often could see how using data could improve practice, but were 
negative about the barriers that made data use hard for them. Many of these barriers related to 
principal leadership and data systems. Some principals in this study were shown to be employing 
multiple leadership strategies to facilitate faculty data use; in those schools, attitudes were good and 
data supported educational practice. Unfortunately, more principals were employing few such 
strategies; in those schools, educators reported negative attitudes, and had difficulty using data to 
improve practice. Results also showed a range of hardships and wasted time caused by inefficient 
computer data systems. Most importantly, these factors together affected how data use translated 
into knowledge and practice. 

Fortunately, many of these barriers are ones that can be handled structurally. Formal district 
policies for using data were largely absent from our results, so we suggest that districts can support 
and facilitate classroom practice by attending to data-related district policy. That is, districts can 
improve how data inform practice if leaders are intentional about writing and codifying district 
policy to alleviate the barriers found in the present study. In the following sections, we provide 
discussion of five areas that district leaders might consider when creating policy: context, attitudes, 
principal leadership, computer data systems, and focus on all learners. 

Context 

Context can be important in considering district data use policies. However, our results 
suggest that, regardless of context, districts may be dealing often with similar policy issues. We 
identified a few findings that were considerably different from district to district; other differences 
existed only in details specific to each district. 

One finding from this study depended strongly on context: the interaction of context and 
policy environment seemed to affect how data were used. Gibson and Musial were average- 
achieving districts looking to improve, so there was a stronger response to accountability ratings in 
these districts than in Boyer. The attention given to accountability ratings in Gibson and Musial 
sometimes seemed to distract principals and other leaders from the bigger picture of teaching and 
learning. Educators in the high-achieving Boyer district did not view their students’ achievement in 
terms of accountability. Still, accountability had an effect on data use in Boyer, because many 
educators saw data use as accountability-related and thus believed that data use did not apply to 
them. Thus, the accountability system created little motivation in Boyer to engage in data use. 

For most other findings, context mattered only in detail. For instance, regardless of district, 
we found that educator attitudes toward data were often ambivalent because of barriers, but the 
barriers themselves often depended on district. As another example, all districts had integration 
problems with their data systems, but the responses to this problem varied by district. 

Attitudes 

Attitudes toward data have been shown by prior research to be important: in exemplary 
contexts, studies have shown attitudes toward data to be positive and educators believed that data 
use positively supported their practice (Wayman & Stringfield, 2006; Datnow et al., 2007; Lachat & 
Smith, 2005). However, studies in these contexts rarely present the daily barriers shown in our data. 
Our data showed attitudes toward data were remarkably positive, even in the face of these barriers. 
Positive perceptions of data’s potential seemed to be what kept educators using data, because many 
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thought data could help them in their practice. Consequently, educators sometimes even constmcted 
time-consuming workarounds in order to use data. 

This notwithstanding, it must not be ignored that educator attitudes in our study often were 
negative toward what data use did to them. At first glance, this seemed related to role: survey data 
indicated that teachers were consistently more negative than administrators or support staff about 
using data. However, in triangulating survey data with interview data, we came to interpret these 
attitudes as related to everyday practice, not role: attitudes toward data were mostly related to the 
barriers and problems educators faced. 

We surmise that teachers were typically more negative than other roles because the nature of 
how they worked with data made them subject to more barriers. To contrast, instructional support 
staff used data frequendy, but were often more positive. Instructional support jobs often carried 
natural structures that enabled them time and other supports, so it is possible instructional support 
staff did not face the day-to-day barriers that teachers did. It is reasonable to conclude that if the 
activities associated with data make it harder for one to do one’s job, then one would be 
understandably negative about data - regardless of role. 

In considering how attitudes might shape district policy, there is a bright spot in these 
results: attitudes seem to be influenced by structural barriers and these structures can be affected by 
district policy. District policies can be written to improve a number of complaints found in our 
study, such as time to use data, learning opportunities, expectations for data use that align with 
improved practice rather than accountability compliance, principal leadership, and data systems. 

Principals 

Numerous studies have suggested that the principal is important to the success of a data 
initiative (e.g., Wayman & Stringfield, 2006; Copland, 2003; Datnow et al., 2007). In the present 
study, the connection between a principal and faculty appeared to greatly affect how data were used; 
this connection seemed to depend on the strategies employed by the principal. In schools where 
principals intentionally employed strategies to support data use to improve student learning, teachers 
typically reported better attitudes and more effective data use. In schools where principals employed 
fewer or negative strategies, attitudes were worse and data use was reported to be less effective. In 
these schools, teachers were often left to deal with data on their own - not just in terms of the 
meaning they drew from it and subsequent change in practice, but also in stmctures, such as time 
and data access. 

District policies can address principal leadership strategies in a number of ways. Some are 
relatively simple. For instance, policies could provide for professional development that helps 
principals learn how to use data-related leadership strategies. Or, policies could contain a list of the 
types of strategies that district leaders believe may be most effective in the district context (see 
Wayman, Spring, Lemke, & Lehr (2012) for a research-based inventory of such strategies). Policies 
such as these may seem obvious, but we saw no evidence that any of our districts had addressed 
principal leadership in any structured way. 

While these policies are an important first step, the current study provides evidence that 
simple policies such as these — those that only mandate action — are not sufficient. Policies must 
additionally provide elaboration and support for how these actions may be implemented. For 
example, our findings highlighted district policies that provided building-level educators time to 
collaborate (e.g., PLCs, common planning time). Despite these structures, principals rarely 
encouraged or required this time to be allocated to analyzing data. In this case, principals were not 
limited by district policy in providing faculty time to use data, but were not fully supported by this 
policy because it only created collaborative time. Thus, even in the presence of these policies, many 
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teachers still reported insufficient time to use data. These policies would have been more effective 
with regard to data use if they had been written to also provide support for what went on in that 
time. For instance, a policy could require that this time be dedicated to data use a certain number of 
days per week and could provide guidance regarding key problems to work on. 

Computer Data Systems 

Results of this study, like others (e.g., Wayman & Stringfield, 2006; Lachat & Smith, 2005; 
Means et al., 2010), make it clear that educators need integrated, one-stop access to the data they 
want to use. Integration was so important to the educators in the present study that many 
improvised time-consuming methods to connect various pieces of data. The results of this study 
additionally showed how ineffective data systems affected attitudes toward data and the progression 
from data to information to practice. 

Clearly, any district that chooses to make data an important part of their educational 
experience should enable some way to integrate data so that educators have one-stop access to all 
the data that are available. But our findings show that the use of these systems is often more 
complex than merely enabling the hardware. For instance, users and districts were sometimes 
applying these systems in ways different than envisioned by system designers. Cho and Wayman 
(2012) suggested a reason for this: system use is dependent on the sense that the user makes of it. 
They further suggest that preconceptions about data systems, data, and their utility for practice serve 
as a lens for determining what is meaningful about a system. . Thus, educators may have certain 
expectations, potential uses, or kinds of practices in mind before accessing data. This may promote 
some system features as worthy of use or adaptation, while obscuring the significance of others. 

Consequently, district policies for data systems should be twofold: Besides providing the 
means for providing one-stop, integrated access, policies must also attend to how the system will fit 
into the everyday work of the educator. We posit that data systems should be at the center of every 
district policy about data use. For instance, a policy dealing with professional learning should outline 
how the data system will be used to support skills educators gain during professional learning 
sessions. Or, a policy supporting principal time structures should also describe how the data system 
may be used to make use of this time more productive. 

Focus on All Learners 

Our results showed data use being focused primarily on struggling learners. We believe this 
is probably the case in most districts - while prior research has not explicitly identified or quantified 
this result as in the current study, almost all of the uses of data described in prior research involve 
struggling learners (e.g., Anderson et al., 2010; Wayman & Stringfield, 2006; Datnow et al., 2007; 
Lachat & Smith, 2005; Marsh et al., 2010; Young, 2006). Focusing on struggling learners is an 
absolute necessity; this fact has been underscored in the current accountability climate. However, we 
also caution against narrowing data use to focus on one group of students, because narrowing 
toward one group does not fit how schools work. Instead, we suggest that a data initiative will be 
more successful for struggling and non-stmggling students alike if it is focused on formative 
opportunities for every student throughout the district. District data policies should be written to 
describe how this focus on all students will raise the achievement of struggling learners through 
increased differentiation or other sound practices. 

Conclusion 

Most of the educators in our study wanted data to be used to support classroom practice. 
Unfortunately, they faced barriers and problems that sometimes made this hard to do. Our personal 
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experience and our reading of the literature indicate that many districts across the United States are 
in the same situation as our study districts. 

While we found that the influences on data use were numerous and complex, we also believe 
that many influences can be made positive by effective district policy. Unfortunately, our districts 
had very few - if any - policies related to data use. On the bright side, some of these policies can be 
easily written. Further, a little bit of policy may go a long way toward improving conditions around 
data use. 

If districts are to write effective policy, they must keep in mind that, at the heart, the findings 
of this study speak to how data use fits (or does not fit) into the everyday work of education. Thus, 
it is likely that district policies grounded in compliance will not be effective, but policies that react to 
and support the ways that educators work will be effective. 

We believe that the effective use of data can be a strong contributor to educational practice. 
However, we believe this hypothesis cannot be confirmed until contexts are created where a variety 
of factors support data use district-wide. This study has presented new information toward making 
these supports a reality and we hope to see such supports become commonplace in the coming 
years. 
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